Image processing apparatus and computer-readable medium

ABSTRACT

There is provided an image processing apparatus. The image processing apparatus includes: a brightness information acquisition unit configured to acquire brightness information indicating brightness of each pixel in a target image, wherein high-frequency components are eliminated from the target image; a correction magnification setting unit configured to set, for each pixel of the target image, a correction magnification based on the brightness information, wherein the correction magnification is substantially inversely proportional to the brightness of the pixel; and a gradation correction unit configured to correct the brightness of each pixel based on the correction magnification.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Japanese Patent Application No.2009-161305, filed on Jul. 8, 2009, the entire contents of which arehereby incorporated by reference.

BACKGROUND

1. Technical Field

The present invention relates to an image processing technique, andparticularly relates to gradation correction of an image.

2. Related Art

JP-A-2003-116049 discloses an exposure control technique in an imagingdevice, in which a degree of backlight effect indicating the intensityof backlight effect is detected using a luminance level in a lowluminance level region and a luminance level in another region, andcontrol parameters (gains) for correcting gradation are set to raise theluminance level in the low luminance level region in accordance with thedegree of backlight effect.

According to JP-A-2003-116049, exposure can be controlled properly evenagainst light, so that good gradation can be ensured in a picked-upimage.

In JP-A-2003-116049, however, control parameters (gains) having one andthe same value are unconditionally applied to a plurality of regionshaving one and the same luminance level in an image regardless of thestate of a subject in each region. Therefore, there occurs a problemthat the contrast in a local region where the brightness variesdrastically may be degraded in an image subjected to gradationcorrection.

SUMMARY OF THE INVENTION

Exemplary embodiments of the present invention address the abovedisadvantages and other disadvantages not described above. However, thepresent invention is not required to overcome the disadvantagesdescribed above, and thus, an exemplary embodiment of the presentinvention may not overcome any of the disadvantages described above.

Accordingly, it is an illustrative aspect of the present invention tokeep the contrast in a local region, where the brightness variesdrastically, in correcting gradation in an image such as a picked-upimage or the like.

According to one or more illustrative aspects of the present invention,there is provided an image processing apparatus. The image processingapparatus includes: a brightness information acquisition unit configuredto acquire brightness information indicating brightness of each pixel ina target image, wherein high-frequency components are eliminated fromthe target image; a correction magnification setting unit configured toset, for each pixel of the target image, a correction magnificationbased on the brightness information, wherein the correctionmagnification is substantially inversely proportional to the brightnessof the pixel; and a gradation correction unit configured to correct thebrightness of each pixel based on the correction magnification.

According to one or more illustrative aspects of the present invention,there is provided a computer-readable medium storing a program forcausing the computer to perform operations including: (a) acquiringbrightness information indicating brightness of each pixel in a targetimage, wherein high-frequency components are eliminated from the targetimage; (b) setting, for each pixel of the target image, a correctionmagnification based on the brightness information, wherein thecorrection magnification is substantially inversely proportional to thebrightness of the pixel; and (c) correcting the brightness of each pixelbased on the correction magnification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a hardware configurationof a digital still camera according to the invention;

FIG. 2 is a functional block diagram of the digital still cameraaccording to the invention;

FIG. 3 is a functional block diagram showing a main portion of an imageprocessor;

FIG. 4 is a functional block diagram showing a configuration of acontrast adjustment unit;

FIG. 5 is a conceptual view showing a function of an ε-filter;

FIG. 6A shows an example of a brightness information image, FIG. 6Bshows an example of an image subjected to processing with the ε-filter,and FIG. 6C shows an example of an image subjected to processing with alow pass filter;

FIGS. 7A and 7B are characteristic graphs showing a change in a contrastadjustment level with respect to a backlight effect level;

FIGS. 8A and 8B are graphs showing the relationship between a firstbrightness component value and a combination ratio of the firstbrightness component value to a third brightness component value;

FIGS. 9A-9C are characteristic graphs showing a change in a gain foreach pixel with respect to a fourth brightness component value;

FIG. 10 is a flow chart showing the contents of processing in the imageprocessor;

FIGS. 11A and 11B are explanatory views of a gain set for each pixel ina target image; and

FIGS. 12A and 12B are views showing the effect of gradation correctionin the image processor.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary embodiments of the present invention will be now describedbelow with reference to the drawings. FIG. 1 is a block diagram showingthe outline of the hardware configuration of a digital still camera(hereinafter referred to as “digital camera” simply) illustrated as anexemplary embodiment of the invention.

As shown in FIG. 1, the digital camera 1 has a CCD (Charge CoupledDevice) 2 for imaging a subject. The CCD 2 has a structure in which acolor filter having a specific color array such as a Bayer array isprovided in a photosensitive surface where an optical image of thesubject should be formed by a not-shown optical system. The CCD 2 isdriven by horizontal and vertical transfer drive signals received from ahorizontal/vertical driver 3, so as to convert the optical image of thesubject into an electric signal. The converted electric signal issupplied as an imaging signal to a CDS/AD circuit 4, including a CDS(correlated Double Sampling) circuit and an A/D converter(Analog-to-Digital converter).

The horizontal/vertical driver 3 operates based on a timing signalgenerated by a TG (Timing Generator) 5, so as to generate the horizontaland vertical transfer drive signals, thereby driving the CCD 2. Thetiming signal generated by the TG 5 is also provided to the CDS/ADcircuit 4. The CDS/AD circuit 4 operates based on the timing signalreceived from the TG 5, so as to remove noise contained in the imagingsignal outputted by the CCD 2, and convert the noise-removed imagingsignal into a digital signal. The converted digital signal is providedto a DSP (Digital Signal Processor) 6.

The DSP 6 is provided with a buffer memory 6 a for processing thedigital signal received from the CDS/AD circuit 4, that is, image datacomposed of data of each pixel having only single color information. TheDSP 6 performs processing as follows. That is, for the image datareceived from the CDS/AD circuit 4, the DSP 6 interpolates missing colorinformation in each pixel based on pixels around the pixel. Thus, theDSP 6 performs de-mosaic processing to generate image data with colorcomponent information of R (Red), G (Green) and B (Blue) for each pixel,that is, RGB data.

In addition, the DSP 6 performs digital signal processings such asgradation correction on the RGB data generated by the de-mosaicprocessing, processing for adjusting white balance, gamma correctionprocessing, various filter processings on the RGB data subjected to thegradation correction, YUV conversion processing for converting the RGBdata into YUV data which are image data expressed by a luminancecomponent (Y) and two color difference components (Cb and Cr) for eachpixel, etc. The details of the gradation correction will be describedlater.

In addition, the DSP 6 supplies the YUV data subjected to the digitalsignal processings, sequentially to an SDRAM (Synchronous DynamicRandom-Access Memory) 7. The SDRAM 7 temporarily stores the YUV datareceived from the DSP 6.

Further, whenever one frame of YUV data (image data) is accumulated inthe SDRAM 7 in a recording mode set as the operating mode of the digitalcamera 1, the DSP 6 will read the image data accumulated in the SDRAM 7and supply the read image data to an LCD (Liquid Crystal Display) 8.

The LCD8 has a not-shown liquid crystal display, a drive circuit fordriving the liquid crystal display, etc. An image based on image datareceived from the DSP 6 is displayed as a live view image on a screen ofthe liquid crystal display.

On the other hand, the TG5 and the DSP 6 are connected to a CPU (CentralProcessing Unit) 9 through a bus 13. The operations of the TG 5 and theDSP 6 are controlled by the CPU 9.

The CPU 9 operates in accordance with a program stored in a flash memory10 serving as an EEPROM (Electric Erasable Programmable Read OnlyMemory) whose memory contents can be rewritten, so as to control theoperation of the digital camera 1 as a whole.

In addition, when an image is picked up in the recording mode set as theoperating mode of the digital camera 1, the CPU 9 compresses image datatemporarily stored in the SDRAM 7 in a predetermined compression systemsuch as a JPEG (Joint Photographic Expert Group) system, and records thecompressed image data as an image file into an external memory 11. Theexternal memory 11 is a memory card which is removably attached to thecamera body and connected to the bus 13 through a not-shown cardinterface.

In addition, when a reproduction mode is set as the operating mode ofthe digital camera 1, the CPU 9 reads a given image file (compressedimage data) from the external memory 11 in accordance with necessity,and expands the read data on the SDRAM 7. Further, the CPU 9 suppliesthe data (YUV data) expanded on the SDRAM 7 to the LCD 8 through the DSP6 to display a recorded image on the LCD 8.

A key input unit 12 is connected to the bus 13. The key input unit 12has various operating keys required for user's operation on the digitalcamera 1, such as a power key, a shutter key, a mode setting key forsetting the recording mode or the reproduction mode, etc. The CPU 9successively detects the operating state of each operating key in thekey input unit 12, and executes various processings along programs inaccordance with a user's request decided by the detected operatingstate.

FIG. 2 is a functional block diagram showing the configuration of thedigital camera 1. As shown in FIG. 2, the digital camera 1 includes: animaging unit 51; an image processor 52, a controller 53, a workingmemory 54, a program memory 55, an image recorder 56, a display unit 57and an operating unit 58.

Each functional block is implemented by one piece or plural pieces ofhardware shown in FIG. 1, as described later. That is, the imaging unit51 is implemented by the CCD 2, the horizontal/vertical driver 3, theCDS/AD circuit 4 and the TG 5 so as to pick up an image of a subject andcapture the picked-up image. The image processor 52 is implemented bythe DSP 6 so as to perform the image processings on the picked-up image.The controller 53 is implemented by the CPU 9. The working memory 54 andthe program memory 55 are implemented by the SDRAM 7 and the flashmemory 10 respectively. The image recorder 56, the display unit 57 andthe operating unit 58 are implemented by the external memory 11, the LCD8 and the key input unit 12 respectively.

FIG. 3 is a functional block diagram showing the details of the imageprocessor 52. Respective units of the image processor 52 correctgradation on RGB data generated by the de-mosaic processing. As shown inFIG. 3, the image processor 52 includes: an image buffer 101; an RGB maxcomputing unit 102; an ε-filter 103; a gain setting processor 104; and agradation correction unit 105. In addition, the gain setting processor104 includes: a backlight effect level computing unit 111; a contrastadjustment level computing unit 112; a contrast adjustment unit 113; acombination processing unit 114, and a corrected gain calculator 115.Further, the contrast adjustment unit 113 includes a subtracter 121, amultiplier 122 and an adder 123, as shown in FIG. 4.

Each unit of the image processor 52 shown in FIGS. 3 and 4 will bedescribed below. The image buffer 101 is implemented by theaforementioned memory 6 a (see FIG. 1). The image buffer 101 stores RGBdata (RGB_in) generated by the de-mosaic processing. The RGB datarepresent an image to be subjected to the gradation correction. In thefollowing description, the image represented by the RGB data will bereferred to as “target image”. Pixel values of R, G and B colorcomponents in the RGB data are in the range “0 to 255”.

The RGB max computing unit 102 serves as a brightness componentextracting unit. The RGB max computing unit 102 reads the RGB datastored in the image buffer 101. For each pixel, the RGB max computingunit 102 selects one of the R, G and B color components which has thehighest pixel value, and acquires the pixel value of the selected colorcomponent as a brightness component of the target image. Then, the RGBmax computing unit 102 provides a first brightness component value (max)to the ε-filter 103, the contrast adjustment unit 113 and thecombination processing unit 114. The first brightness component value(max) is a pixel value of a color component whose pixel value is thehighest of all the acquired pixel values in each pixel.

The ε-filter 103 is an image filter which mainly serves to eliminatesmall-amplitude noise (high-frequency component) overlaid on an imagewith a sharp change of brightness, that is, to smooth the brightness ofthe image. Particularly, the ε-filter 103 is a smoothing filter capableof holding edges of an original image. The ε-filter 103 serves as asmoothing unit, which performs filtering processing (described later) toconvert the first brightness component value (max) of each pixelreceived from the RGB max computing unit 102 into a second brightnesscomponent value (max_ε). Then, the ε-filter 103 provides the convertedsecond brightness component value (max_ε) to the contrast adjustmentunit 113 in the gain setting processor 104.

The RGB max computing unit 102 and the ε-filter 103 serve as abrightness information acquisition unit.

Here, the filtering processing in the ε-filter 103 will be nowdescribed. In the ε-filter 103, each pixel is set as a pixel of interestand attention is paid to a pixel region of 3 pixels square (that is, aregion including 9 pixels with the pixel of interest as a center of thepixel region). That is, attention is paid to the pixel of interest andeight peripheral pixels around the pixel of interest. Pixel values ofthe peripheral pixels are adjusted so that a differential value betweenthe pixel value (first brightness component value) of the pixel ofinterest and the pixel value (first brightness component value) of eachperipheral pixel can be made not higher than a threshold T (T=20).Further, the original pixel value of the pixel of interest and theadjusted pixel values of the peripheral pixels are multiplied by 1/9 asa predetermined coefficient, and the total sum of the pixel valuesobtained thus is calculated. The calculated pixel value is set as a newpixel value (second brightness component value) of the pixel ofinterest.

In the aforementioned filtering processing, the pixel value (firstbrightness component value) of the pixel of interest is increased ordecreased in accordance with the pixel values (first brightnesscomponent values) of the peripheral pixels so that the pixel values ofpixels adjacent to each other can be averaged in the image. In addition,in the course of the filtering processing, the pixel value of eachperipheral pixel is adjusted to make a differential value from the pixelvalue of the pixel of interest not higher than the threshold T, and thepixel value of the pixel of interest is increased or decreased(averaged) with the adjusted pixel values of the peripheral pixels.

Thus, in the aforementioned filtering processing, it is possible toreduce the influence which the pixel values of the peripheral pixelslocated on the bright region side of a border portion between a brightregion and a dark region in an image (for example, a border portion of asubject such as a person or a building) gives to the pixel value of thepixel of interest located on the dark region side. Similarly, in theaforementioned filtering processing, it is possible to reduce theinfluence which the pixel values of the peripheral pixels located on thedark region side of a border portion between a bright region and a darkregion in an image gives to the pixel value of the pixel of interestlocated on the bright region side.

Thus, in the filtering processing in the ε-filter 103, brightness ineach portion of the original image can be smoothed while the edges inthe original image are kept. In the filtering processing, it is possibleto suitably change the range of the pixel region of interest, the valueof the threshold T, and the coefficient by which the pixel value of eachpixel should be multiplied.

FIG. 5 is a conceptual view showing the function of the ε-filter 103.That is, FIG. 5 is a view showing a change in brightness of each pixelwith respect to the position of the pixel in a border portion between abright region and a dark region in an image. In the border portion,brightness varies drastically. In FIG. 5, the abscissa represents ahorizontal (or vertical) pixel position, and the ordinate representsbrightness of a pixel in each pixel position. The change in brightnessdepicted by the solid line represents a change in a first brightnesscomponent value (max) before the filtering processing. The change inbrightness depicted by the broken line represents a change in a secondbrightness component value (max_ε) after the filtering processing.

FIGS. 6A-6C are views showing specific examples of the effect of thefiltering processing by the ε-filter 103. FIG. 6A shows an example of abrightness component image composed of pixels whose pixel values arefirst brightness component values (max) which have not been subjected tothe filtering processing. FIG. 6B shows an example of a global luminanceimage composed of pixels whose pixel values are second brightnesscomponent values (max_ε) which have been subjected to the filteringprocessing, that is, an image where high-frequency components have beeneliminated from the brightness component image. FIG. 6C shows an exampleof an image obtained when typical filtering processing (smoothingprocessing) with an LPF (Low Pass Filter) is performed on the brightnesscomponent image in FIG. 6A.

As is apparent from FIGS. 6A-6C, in the global luminance image subjectedto the filtering processing by the ε-filter 103, low-frequencycomponents included in the brightness component image before theprocessing have been reflected (or edges have been kept) more accuratelythan in the image obtained by the typical smoothing processing with theLPF.

The gain setting processor 104 serves as a correction magnificationsetting unit to individually acquire gains (g_lev) as correctionmagnifications for correcting gradation for pixels of a target imagebased on the second brightness component values (max_ε) received fromthe ε-filter 103, and provide (set) the acquired gains (g_lev) to thegradation correction unit 105. Description will be made below on thedetails of each aforementioned functional block of the gain settingprocessor 104 and a specific method for acquiring the gains (g_lev) forcorrecting gradation.

The backlight effect level computing unit 111 serves as a backlighteffect degree acquisition unit to read RGB data stored in the imagebuffer 101 and acquire a backlight effect level (gk_lev) indicating thedegree of backlight effect in a target image based on the read RGB data.The backlight effect level computing unit 111 provides the acquiredbacklight effect level (gk_lev) to the contrast adjustment levelcalculator 112. The backlight effect level computing unit 111 acquiresthe backlight effect level (gk_lev) with the following method.

That is, the backlight effect level computing unit 111 divides a targetimage into n×m regions, and obtains a luminance level (e.g., an averageor a sum of luminance values of pixels in each region) based on RGB datafor each region. Next, the backlight effect level computing unit 111computes a difference between a luminance level of a specific regionwhose luminance level is the lowest and the average value of luminancelevels in the other regions than the specific region, and obtains thecomputed difference in luminance level (having a value ranging from 0 to255) as a backlight effect level (gk_lev). Any specific method forobtaining the backlight effect level in a target image is not limited tothe aforementioned method.

The contrast adjustment level calculator 112 serves as an adjustmentvalue acquisition unit to acquire a contrast adjustment level (adj_lev)in accordance with the degree of backlight effect of the target image,and provides the contrast adjustment level (adj_lev) to the contrastadjustment unit 113. Specifically, the contrast adjustment levelcalculator 112 calculates the contrast adjustment level (adj_lev) usinga predetermined contrast adjustment level computing function, which usesthe backlight effect level (gk_lev) received from the backlight effectlevel computing unit 111 as a parameter.

According to the adjustment level computing function used forcalculating the contrast adjustment level (adj_lev) by the contrastadjustment level calculator 112, the contrast adjustment level (adj_lev)increases in proportion to the backlight effect level (gk_lev). Morespecifically, the contrast adjustment level (adj_lev) varies inaccordance with the change in the backlight effect level (gk_lev), forexample, as shown in FIG. 7A or 7B. In addition, the contrast adjustmentlevel (adj_lev) calculated by the contrast adjustment level calculator112 is in a range of from 0 to 4.

The contrast adjustment unit 113 serves as a correction unit togetherwith the backlight effect level computing unit 111, the contrastadjustment level calculator 112 and the combination processing unit 114,to correct the second brightness component value (max_ε) received fromthe ε-filter 103 in accordance with the degree of backlight effect inthe target image. The contrast adjustment unit 113 supplies a correctedthird brightness component value (adj) to the combination processingunit 114.

Specifically, the contrast adjustment unit 113 includes: the subtracter121 serving as a subtractor, the multiplier 122 serving as a multiplier,and the adder 123 serving as an adder (see FIG. 4). In the contrastadjustment unit 131, the subtracter 121 subtracts the first brightnesscomponent value (max) from the second brightness component value (max_ε)in each pixel, the multiplier 122 multiplies the subtraction result bythe contrast adjustment level (adj_lev), and the adder 123 adds thefirst brightness component value (max) to the multiplication result.Thus, the aforementioned third brightness component value (adj) isgenerated for each pixel.

Here, the third brightness component value (adj) generated by thecontrast adjustment unit 113 is given by the following Expression (1).

adj=adj_lev×(max_ε−max)+max   (1)

That is, the contrast adjustment unit 113 generates the third brightnesscomponent value (adj) by adding a value, which is obtained bymultiplying the difference (max_ε−max) between the second brightnesscomponent value (max_ε) and the first brightness component value (max)by adj_lev (0 to 4), to the first brightness component value (max).

In addition, the combination processing unit 114 combines the thirdbrightness component value (adj) of each pixel received from thecontrast adjustment unit 113 and the first brightness component value(max) of the pixel received from the RGB max computing unit 102 at apredetermined combination ratio (described later). The combinationprocessing unit 114 provides a combined fourth brightness componentvalue (mix) obtained thus to the corrected gain calculator 115.

Specifically, the combination processing unit 114 first serves as acombination ratio determination unit to calculate a combination ratio ofthe first brightness component value (max) to the third brightnesscomponent value (adj) based on a predetermined combination ratiocomputing function using the first brightness component value (max) ofeach pixel as a parameter. According to the combination ratio computingfunction, a combination ratio (α) of the first brightness componentvalue (max) increases in proportion to the first brightness componentvalue (max) as shown in FIG. 8A.

Then, the combination processing unit 114 serves as a combination unitto combine the third brightness component value (adj) and the firstbrightness component value (max) of each pixel according to thefollowing Expression (2) using the combination ratio (α) calculatedbased on the combination ratio computing function:

Mix=α×max+(1−α)×adj   (2)

and provide the combined fourth brightness component value (mix) to thecorrected gain calculator 115.

In addition, the corrected gain calculator 115 computes a gain (g_lev),which should be used for correcting the brightness of each pixel in atarget image, for each pixel according to a predetermined gain computingfunction given by the following predetermined expression and using thefourth brightness component value (mix) of each pixel received from thecombination processing unit 114 as a parameter.

f_gain(mix)

The corrected gain calculator 115 then provides the computing result tothe gradation correction unit 105.

The aforementioned gain computing function used for computing the gain(g_lev) of each pixel by the corrected gain calculator 115 is a functionhaving a correction characteristic in which the gain (g_lev) increaseswith the decrease of the fourth brightness component value (mix) and thegain (g_lev) decreases with the increase of the fourth brightnesscomponent value (mix), that is, a correction characteristic in which thegain (g_lev) is in inverse proportion to the fourth brightness componentvalue (mix), as shown in FIG. 9A.

The gradation correction unit 105 reads RGB data stored in the imagebuffer 101 pixel by pixel, and performs correction thereon bymultiplying a pixel value (R value, G value or B value) of a colorcomponent in RGB data of each pixel in the read target image by the gain(g_lev) for the pixel received from the gain setting processor 104. Inaddition, the gradation correction unit 105 rewrites the RGB data ofeach pixel stored in the image buffer 101 with the corrected RGB data.That is, in the image processor 52, the gradation correction unit 105serves as a gradation correction unit to correct gradation of a targetimage by rewriting RGB data (RGB_in) stored in the image buffer 101 withcorrected RGB data.

Then, the image processor 52 reads corrected RGB data (RGB_out) storedin the image buffer 101, and performs another digital signal processingsuch as white balance adjustment processing on the read corrected RGBdata, that is, an image subjected to gradation correction, as a targetto be processed.

FIG. 10 is a flow chart showing the contents of gradation correction tobe performed by the respective units of the image processor 52 shown inFIG. 3. The details of processing in each step have been alreadydescribed, and thus the description thereof will be omitted herein.

As shown in FIG. 10, in the gradation correction in the image processor52, the RGB max computing unit 102 acquires a first brightness componentvalue (max) of each pixel in a target image from RGB data representingthe target image (Step S1). Next, the ε-filter 103 performs filteringprocessing to convert the first brightness component value (max) into asecond brightness component value (max_ε) (Step S2).

In addition, the backlight effect level computing unit 111 acquires abacklight effect level (gk_lev) from the RGB data representing thetarget image (Step S3). Further, the contrast adjustment levelcalculator 112 calculates a contrast adjustment level (adj_lev)proportional to the backlight effect level (gk_lev) (Step S4). Next, thecontrast adjustment unit 113 generates a third brightness componentvalue (adj) for each pixel based on the first brightness component value(max), the second brightness component value (max_ε) and the contrastadjustment level (adj_lev) (Step S5).

Next, the combination processing unit 114 combines the third brightnesscomponent value (adj) with the first brightness component value at acombination ratio proportional to the size thereof so as to generate afourth brightness component value (mix) (Step S6). Next, the correctedgain calculator 115 calculates a gain (g_lev) for each pixel inaccordance with the fourth brightness component value (mix) (Step S7).After that, the gradation correction unit 105 multiplies the pixel valueof each color component by the gain (g_lev) of each pixel so as tocorrect the RGB data of the pixel (Step S8). Further, the gradationcorrection unit 105 rewrites the RGB data (RGB_in) stored in the imagebuffer 101 with the corrected RGB data of each pixel and stores thecorrected RGB data of the pixel into the image buffer 101 (Step S9).

In correcting gradation by the image processor 52 as described above,the fourth brightness component value (mix) of each pixel provided tothe corrected gain calculator 115 is brightness information based on thefirst brightness component value (max) acquired by the RGB max computingunit 102 and the second brightness component value (max_ε) acquired bythe ε-filter 103. That is, the fourth brightness component value (mix)is based not on brightness information of each pixel in the target imagebut on brightness information of each pixel where high-frequencycomponents have been eliminated from the target image. In addition, thecorrected gain calculator 115 sets a gain in inverse proportion to thefourth brightness component value (mix) of each pixel as a gain (g_lev)of the pixel.

Thus, in correcting gradation by the image processor 52, globalgradation of the target image can be corrected. In addition, in theimage subjected to the gradation correction, similar contrast to that inthe target image can be secured even in a local region having a drasticchange of brightness in the target image.

The reason why the above effect can be obtained will be describedspecifically. First, in correcting gradation by the image processor 52,the gain (g_lev) set for each pixel is in inverse proportion to thefourth brightness component value (mix) of the pixel as described above.Thus, the gain (g_lev) for a pixel becomes higher as the pixel belongsto a dark region in a target image, and the gain (g_lev) for a pixelbecomes lower as the pixel belongs to a bright region in the targetimage. Therefore, in an image subjected to the gradation correction,brightness will be largely increased in a region which has been dark inthe target image.

Thus, the following effect can be basically obtained in the gradationcorrection by the image processor 52. That is, it is possible to securegood gradation in a region which has been dark in the target image whilesuppressing the occurrence of a “whiteout” phenomenon in a region whichhas been bright in the target image, so that global gradation correctioncan be performed on the target image. As a result, for example, when thetarget image is an image where a person is photographed against lightand a face portion of the person is dark, an image where the faceportion of the person is expressed brightly and colorfully can beobtained by the gradation correction by the image processor 52.

On the other hand, when the brightness of pixels in a dark region in atarget image is increased in the gradation correction, the difference inbrightness between a pixel on the dark region side and a pixel on thebright region side is reduced in a local region where the brightness hasvaried drastically in the target image. As a result, the contrast isdegraded, for example, in a portion where a dark subject with acomparatively small area has been against a bright region (such as thesky) used as the background in the target image.

Meanwhile, in the gradation correction by the image processor 52, thefourth brightness component value (mix) of each pixel provided to thecorrected gain calculator 115 is brightness information generated basedon the second brightness component value (max_ε). More specifically, thefourth brightness component value (mix) is brightness information wherethe second brightness component value (max_ε) has been corrected throughthe contrast adjustment unit 113 and the combination processing unit114. Then, the second brightness component value (max_ε) is a pixelvalue of a pixel of a global luminance image where high-frequencycomponents have been eliminated from a brightness component imagecomposed of pixels whose pixel values are first brightness componentvalues (max) as described above that is, a global luminance imagecomposed of only low-frequency components.

Thus, a gain (g_lev) of each pixel calculated by the corrected gaincalculator 115 in the gradation correction, that is, a gain set for eachpixel of a target image is characterized as follows, as compared withthe case where the first brightness component value (max) is provided tothe corrected gain calculator 115. For the sake of convenience ofexplanation, the details of a gain set for each pixel of a target imagewill be described with comparison between the case where the firstbrightness component value (max) is provided to the corrected gaincalculator 115 and the aforementioned case where the second brightnesscomponent value (max_ε) which has not been corrected is provided to thecorrected gain calculator 115.

FIGS. 11A and 11B are graphs showing a difference in gain set for eachpixel in a target image between the case where the first brightnesscomponent value (max) is provided to the corrected gain calculator 115and the case where the second brightness component value (max_ε) whichhas not been corrected is provided to the corrected gain calculator 115.

That is, FIG. 11A is a graph showing an example of a gain set for adarker pixel than its peripheral pixels in a target image. In a darkerpixel than its peripheral pixels, the second brightness component value(max_ε) of the darker pixel than the peripheral pixels becomes largerthan its original first brightness component value (max) because theoriginal first brightness component value (max) is smoothed with thefirst brightness component values (max) of the peripheral pixels.Therefore, the gain obtained when the second brightness component value(max_ε) is provided to the corrected gain calculator 115 becomes lowerthan the gain obtained when the first brightness component value (max)is provided to the corrected gain calculator 115. Thus, the increase ofbrightness caused by the gradation correction is suppressed in thedarker pixel than the peripheral pixels. As a result, in a local regionwhere the brightness has varied drastically in the target image,brightness in a dark region is suppressed close to brightness where thegradation correction has not been performed, as compared with any otherdark region after the gradation correction.

In addition, FIG. 11B is a graph showing an example of a gain set for abrighter pixel than its peripheral pixels in a target image. In abrighter pixel than its peripheral pixels, the second brightnesscomponent value (max_ε) of the brighter pixel than the peripheral pixelsbecomes smaller than its original first brightness component value (max)because the original first brightness component value (max) is smoothedwith the first brightness component values (max) of the peripheralpixels. Therefore, the gain obtained when the second brightnesscomponent value (max_ε) is provided to the corrected gain calculator 115becomes higher than the gain obtained when the first brightnesscomponent value (max) is provided to the corrected gain calculator 115.Thus, the increase of brightness caused by the gradation correctionbecomes conspicuous in the brighter pixel than the peripheral pixels. Asa result, in a local region where the brightness has varied drasticallyin the target image, brightness in a bright region is emphasized ascompared with any other bright region after the gradation correction.

When the brightness of each pixel which is a dark pixel or a brightpixel in a target image is the same as the brightness of peripheralpixels adjacent thereto, the first brightness component value (max) isequal to the second brightness component value (max_ε) in each pixel.Thus, there is less possibility that the contrast is degraded in anyother region than a local region where the brightness variesdrastically.

The details of the gain set for each pixel in the target image in thecase where the second brightness component value (max_ε) is provided tothe corrected gain calculator 115 as described above can be also appliedto the configuration where the fourth brightness component value (mix)is provided to the corrected gain calculator 115.

Thus, in the gradation correction by the image processor 52, goodcontract condition can be obtained in an image subjected to thegradation correction even if the brightness is increased in pixels of adark region in a target image by the gradation correction. That is,similar contrast to that before the gradation correction can be securedin the image subjected to the gradation correction in a local regionwhere the brightness has varied drastically in the target image.

FIGS. 12A and 12B are views specifically showing the effect of theaforementioned gradation correction. That is, FIG. 12A is a viewpartially showing an image subjected to the gradation correction when atarget image is represented by the first brightness component values(max) as shown in FIG. 6A. More specifically, FIG. 12A is a view showingan upper right portion of the image subjected to the gradationcorrection when the image is equally divided into four vertically andhorizontally. On the other hand, FIG. 12B is a view showing acomparative example. FIG. 12B partially shows an image in whichrelated-art gradation correction to increase the luminance level of alow luminance level region simply has been performed on the targetimage, corresponding to FIG. 12A.

As shown in FIG. 12B, when the luminance level of the low luminancelevel region is increased simply according to the related-art gradationcorrection, the contrast will deteriorate in a region where thin anddark branches and leaves are complicated against the bright sky asillustrated in FIG. 12B, that is, a local region where the brightnessvaries drastically. On the other hand, according to the gradationcorrection by the image processor 52, as shown in FIG. 12A, due to theaforementioned reason, the contrast is prevented from deteriorating evenin a region where thin and dark branches and leaves are complicatedagainst the bright sky, so that the contrast similar to that of thetarget image can be secured.

As described above, according to the gradation correction by the imageprocessor 52, global gradation of the target image which has not beensubjected to the gradation correction is corrected, while the similarcontrast to that of the target image can be secured in an image whichhas been subjected to the gradation correction, even in a local regionwhere the brightness has varied drastically in the target image whichhas not been subjected to the gradation correction.

In addition, according to the gradation correction by the imageprocessor 52, RGB data generated by de-mosaic processing is used asimage data to be processed, and a pixel value (R value, G value or Bvalue) of each color component in each pixel is corrected individuallyin accordance with a correction gain (g_lev) set for the pixel, so as tocorrect the brightness of the pixel. Thus, the following effect can beobtained.

That is, when the pixel value (R value, G value or B value) of eachcolor component is multiplied by the gain (g_lev) set for each pixel inthe target image to increase the pixel value of the color component, thecolor saturation of the image subjected to the gradation correction isincreased. This is because color saturation (S) of each pixel isproportional to the difference between a value (MAX) of a colorcomponent whose pixel value is the highest of the pixel values (R value,G value and B value) of the color components and a value (MIN) of acolor component whose pixel value is the lowest thereof(S=(MAX−MIN)/MAX).

On the other hand, assuming that YUV data after YUV conversionprocessing is used as image data to be processed in the gradationcorrection and a luminance component value (Y value) of each pixel ismultiplied by a correction gain to increase the luminance, the colorsaturation of the image which has been subjected to the gradationcorrection is not increased. This is because the increase/decrease ofthe luminance component value (Y value) gives no influence to theincrease/decrease of the color saturation.

As described above, according to the gradation correction by the imageprocessor 52, high color saturation can be secured in an image which hasbeen subjected to the gradation correction, so that a plurality ofcolors can be distinguished easily on the image which has been subjectedto the gradation correction. Thus, the image which has been subjected tothe gradation correction can be made vivid. In addition, original colorscan be reproduced accurately in the image which has been subjected tothe gradation correction, even in a dark pixel for which a high gain(g_lev) is set in the gradation correction, that is, a pixel whosebrightness correction intensity is high.

On the other hand, the image processor 52 acquires the second brightnesscomponent value (max_ε) as a source of the fourth brightness componentvalue (mix) which is a factor of the gain (g_lev) set for each pixel inthe target image, that is, brightness information of each pixel in whichhigh-frequency components have been eliminated from the target image(hereinafter referred to as “low-frequency component information”) inthe following manner. That is, as described above, the image processor52 once acquires first brightness component values (max) from RGB datarepresenting the target image, and performs smoothing processing withthe ε-filter 103 on a brightness component image using the acquiredfirst brightness component values (max) as pixel values, so as toacquire second brightness component values (max_ε).

Thus, the image processor 52 can acquire the low-frequency componentinformation of the target image efficiently. That is, the low-frequencycomponent information of the target image can be also obtained such thatsmoothing processing is performed on RGB data by the ε-filter 103 andbrightness component values of respective pixels are extracted from theprocessed RGB data. However, the volume of data to be processed insmoothing processing is large when the smoothing processing is performedon RGB data by the ε-filter 103. In comparison with this, the volume ofdata to be processed in smoothing processing is much smaller when firstbrightness component values (max) are obtained from RGB data andsmoothing processing is performed on the obtained first brightnesscomponent values (max) by the ε-filter 103. Therefore, the imageprocessor 52 can acquire the low-frequency component information of thetarget image efficiently.

In addition, in the image processor 52, the ε-filter 103 is a smoothingfilter having edge holding performance. Thus, in an image subjected tothe gradation correction, similar contrast to that of the target imagecan be reproduced accurately even in a border portion between a brightregion and a dark region with a large difference in brightness betweenthe bright region and the dark region correspondingly to the outline ofa subject such as a person or a building occupying a comparatively largearea in the target image. The reason why such an effect can be obtainedwill be described below. Here, the effect obtained using the ε-filter103 in the image processor 52 will be described in comparison with thecase where a typical low pass filter having no edge holding performanceis used in the image processor 52.

Firstly, when a typical low pass filter is used in the image processor52, the degree of blurring appearing in the border portion between abright region and a dark region becomes larger in a global luminanceimage subjected to the filtering processing (smoothing processing) thanwhen the ε-filter 103 is used, as shown in FIG. 6C. That is, the globalluminance image is obtained as an image in which brightness near theborder on the bright region side in the border portion between thebright region and dark region before the filtering processing is darker(with a smaller pixel value) than brightness in the bright regiondistant from the border between the bright region and the dark region,and brightness near the border on the dark region side in the borderportion between the bright region and dark region before the filteringprocessing is brighter (with a large pixel value) than brightness in thedark region distant from the border between the bright region and thedark region.

On the other hand, in the gradation correction by the image processor52, as described above, the gain (g_lev) set for each pixel is ininverse proportion to the fourth brightness component value (mix) of thepixel provided to the corrected gain calculator 115. Therefore, when atypical low pass filter is used in the image processor 52, a gain is setfor each pixel forming the border portion between the bright region andthe dark region in the target image as follows. That is, on the brightregion side in the border portion between the bright region and the darkregion, the gain set for each pixel forming the vicinity of the borderbetween the bright region and the dark region is larger than that foreach pixel in a region distant from the border between the bright regionand the dark region. In addition, on the dark region side in the borderportion between the bright region and the dark region, the gain set foreach pixel forming the vicinity of the border between the bright regionand the dark region is smaller than that for each pixel in a regiondistant from the border between the bright region and the dark region.

Accordingly, when there is a poor change in brightness of each pixel ina bright region side region or a dark region side region forming theborder portion between the bright region and the dark region andcorresponding to the outline of a subject in the target image, thebrightness in the border portion between the bright region and the darkregion after the gradation correction will be described below. That is,there occurs a large difference between brightness of each pixel in aregion which is located in the border portion between the bright regionand the dark region and close to the border between the bright regionand the dark region and brightness of each pixel in a region which isdistant from the border between the bright region and the dark region,in spite of a small variation in brightness therebetween in the targetimage. In addition, the larger the difference in brightness between thebright region side and the dark region side in the border portion is,the more conspicuous such a difference in brightness is.

For the above reason, when a typical low pass filter is used in theimage processor 52, similar contrast to that of the target image cannotbe secured accurately, in an image subjected to the gradationcorrection, in the border portion between the bright region and the darkregion which has a large difference in brightness and corresponds to theoutline of a subject such as a person or a building occupying acomparatively large area in the target image.

On the other hand, in the configuration where the ε-filter 103 is usedin the image processor 52 as in this exemplary embodiment, the degree ofblurring appearing in the border portion between the bright region andthe dark region can be made lower in a global luminance image subjectedto the filtering processing (smoothing processing) than when a typicallow pass filter is used, as shown in FIG. 6B. That is, an image which iskept bright (with a large pixel value) near the border on the brightregion side in the border portion between the bright region and the darkregion which has not been subjected to the filtering processing and dark(with a small pixel value) near the border on the dark region side canbe obtained as a global luminance image.

Thus, in using the ε-filter 103 in the image processor 52, a gain is setfor each pixel forming the border portion between the bright region andthe dark region in the target image when the gradation correction isperformed by the image processor 52. That is, a gain set for each pixelforming the vicinity of the border on the bright region side in theborder portion between the bright region and the dark region issubstantially similarly set for each pixel in a region distant from theborder. In addition, a gain set for each pixel forming the vicinity ofthe border on the dark region side in the border portion between thebright region and the dark region is substantially similarly set foreach pixel in a region distant from the border.

Accordingly, even when there is a poor change in brightness of eachpixel in a bright region side region or a dark region side regionforming the border portion between the bright region and the dark regionand corresponding to the outline of a subject in the target image,similar brightness can be kept between each pixel in a region close tothe border between the bright region and the dark region and each pixelin a region distant from the border between the bright region and thedark region regardless of the bright region side or the dark region sidewhere there has been a poor variation in brightness in an imagesubjected to the gradation correction.

For the above reason, when the ε-filter 103 is used in the imageprocessor 52, similar contrast to that of the target image can bereproduced accurately, in an image subjected to the gradationcorrection, even in the border portion between the bright region and thedark region which has a large difference in brightness and correspondsto the outline of a subject such as a person or a building occupying acomparatively large area in the target image.

In addition, according to the gradation correction in the imageprocessor 52, the following effect can be obtained in addition to theabove effect. That is, in the gradation correction, the secondbrightness component value (max_ε) generated in the ε-filter 103 iscorrected for each pixel in accordance with the first brightnesscomponent value (max) in the contrast adjustment unit 113. Then, thegain (g_lev) of each pixel in the target image is determined based onthe corrected third brightness component value (adj). As a result,brightness/darkness balance among pixels in the target image which hasnot been subjected to the gradation correction can be reflected onglobal gradation of an image which has been subjected to the gradationcorrection. Thus, natural gradation condition can be secured in theimage which has been subjected to the gradation correction.

Further, according to the gradation correction in the image processor52, the following effect can be obtained in addition to the aboveeffects. That is, in the gradation correction, the fourth brightnesscomponent value (mix) of each pixel provided to the corrected gaincalculator 115 is brightness information in which the first brightnesscomponent value (max) is combined with the third brightness componentvalue (adj), which is the corrected brightness information of the secondbrightness component value (max_ε), at a predetermined combinationratio.

As a result, according to the gradation correction, it is possible todecrease the probability of the phenomenon where the gradationcorrection leads to “whiteout” condition in pixels which have beenbrighter than their peripheral pixels in the target image. The reasonwhy such an effect can be obtained will be described below. For the sakeof convenience of explanation, description will be made on theassumption that the above contrast adjustment level (adj_lev) with whichthe contrast adjustment unit 113 should correct the second brightnesscomponent value (max_ε) is “1”, and the third brightness component value(adj) coincides with the second brightness component value (max_ε).

As described above, the second brightness component value (max_ε) of apixel which has been brighter than its peripheral pixels in the targetimage becomes smaller than the first brightness component value (max).Accordingly, if the second brightness component value (max_ε) isprovided to the corrected gain calculator 115 as it is, the gain (g_lev)set for the pixel which has been brighter than its peripheral pixels inthe target image will have a large value. This is because the correctedgain calculator 115 sets a gain which is in inverse proportion to thesecond brightness component value (max_ε).

Accordingly, if the second brightness component value (max_ε) isprovided to the corrected gain calculator 115 as it is, the “whiteout”phenomenon will occur easily in the pixels which have been brighter thantheir peripheral pixels in the target image. That is, the probability ofthe “whiteout” phenomenon appearing in the pixels, which have beenbrighter than their peripheral pixels in the target image, in the imagesubjected to the gradation correction will increase inevitably at thecost with which similar contrast to that before the gradation correctionis secured in a local region where brightness has varied drastically inthe target image.

On the other hand, in the gradation correction, the fourth brightnesscomponent value (mix) is provided to the corrected gain calculator 115.The fourth brightness component value (mix) is obtained by the firstbrightness component value (max) combined with the second brightnesscomponent value (max_ε) at a predetermined combination ratio (α). Thatis, the fourth brightness component value (mix) becomes larger than thesecond brightness component value (max_ε) correspondingly to the firstbrightness component value (max) combined therewith at the predeterminedcombination ratio (α). Accordingly, when the fourth brightness componentvalue (mix) is provided to the corrected gain calculator 115, the gain(g_lev) set for each pixel which has been brighter than its peripheralpixels in the target image takes a smaller value than when the secondbrightness component value (max_ε) is provided to the corrected gaincalculator 115 as it is.

According to the gradation correction, therefore, similar contrast tothat before the gradation correction can be secured in a local regionwhere brightness has varied drastically in the target image, while thefourth brightness component value (mix) is provided to the correctedgain calculator 115. Thus, it is possible to decrease the probability ofthe “whiteout” phenomenon appearing in the pixels, which have beenbrighter than their peripheral pixels in the target image, in the imagesubjected to the gradation correction.

In addition, in the gradation correction, the combination processingunit 114 sets the combination ratio, with which the first brightnesscomponent value (max) should be combined with the third brightnesscomponent value (adj), as a combination ratio (α) which is proportionalto the first brightness component value (max). Accordingly, thecombination ratio of the first brightness component value (max) to thethird brightness component value (adj) can be adjusted to be a properratio. As a result, it is possible to surely decrease the probability ofthe “whiteout” phenomenon appearing in the pixels, which have beenbrighter than their peripheral pixels in the target image, in the imagesubjected to the gradation correction.

In addition, according to the gradation correction in the imageprocessor 52, the following effect can be further obtained in additionto the above effects. That is, in the image processor 52, the contrastadjustment unit 113 corrects the second brightness component value(max_ε) to generate the third brightness component value (adj). Asdescribed above, the contrast adjustment unit 113 multiplies thedifference (max_ε−max) between the second brightness component value(max_ε) and the first brightness component value (max) by adj_lev andadds the obtained value to the first brightness component value (max) togenerate the third brightness component value (adj).

The relationship between the third brightness component value (adj)generated by the contrast adjustment unit 113 and the contrastadjustment level (adj_lev) will be described below. First, therelationship between the third brightness component value (adj) and thecontrast adjustment level (adj_lev) will be described about a pixelwhich is brighter, that is, larger in the first brightness componentvalue (max), than its peripheral pixels in a target image. That is, thepixel which is brighter than its peripheral pixels is a pixel where thefirst brightness component value (max) shown by the solid line in FIG. 5(described above) is larger than the second brightness component value(max E) shown by the broken line in FIG. 5. Accordingly, in the pixelwhich is brighter than its peripheral pixels, the difference (max_ε−max)between the second brightness component value (max_ε) and the firstbrightness component value (max) takes a negative value. Thus, as thevalue of the contrast adjustment level (adj_lev) increases, the thirdbrightness component value (adj) decreases with reference to the secondbrightness component value (max_ε).

Next, the relationship between the third brightness component value(adj) and the contrast adjustment level (adj_lev) will be described on apixel which is darker, that is, smaller in the first brightnesscomponent value (max), than its peripheral pixels in a target image.That is, the pixel which is darker than its peripheral pixels is a pixelwhere the first brightness component value (max) shown by the solid linein FIG. 5 is smaller than the second brightness component value (max_ε)shown by the broken line in FIG. 5. Accordingly, in the pixel which isdarker than its peripheral pixels, the difference (max_ε−max) betweenthe second brightness component value (max_ε) and the first brightnesscomponent value (max) takes a positive value. Thus, as the value of thecontrast adjustment level (adj_lev) increases, the third brightnesscomponent value (adj) increases with reference to the second brightnesscomponent value (max_ε).

On the other hand, in the image processor 52, the corrected gaincalculator 115 sets a gain inversely proportional to the fourthbrightness component value (mix) of each pixel as a gain (g_lev) of thepixel. According to the gradation correction in the image processor 52,if the contrast adjustment level (adj_lev) is increased, each pixel thathas been brighter than its peripheral pixels in the target image will bebrighter after the gradation correction, and each pixel that has beendarker than its peripheral pixels will be darker after the gradationcorrection.

Accordingly, in the gradation correction in the image processor 52, withincrease of the contrast adjustment level (adj_lev), the contrastincreases in an image subjected to the gradation correction. That is, inthe image processor 52, the contrast in the image subjected to thegradation correction can be adjusted by adjustment of the contrastadjustment level (adj_lev).

Further, in the image processor 52, the contrast adjustment level(adj_lev) calculated by the contrast adjustment level calculator 112takes a value increasing in proportion to the backlight effect level(gk_lev). That is, as the degree of backlight effect increases in thetarget image, the contrast adjustment level (adj_lev) becomes larger.Accordingly, in the gradation correction by the image processor 52, thecontrast in the image subjected to the gradation correction can beincreased as the degree of backlight effect increases in the targetimage. As a result, when the target image is an image picked up againstlight, more proper contrast can be secured all over the image subjectedto the gradation correction.

Next, description will be made on modifications of the exemplaryembodiment of the present invention. First of all, in the gradationcorrection by the image processor 52, assume that image data to beprocessed is RGB data generated by de-mosaic processing. However, evenif the image data to be processed by the gradation correction is YUVdata subjected to YUV conversion processing, similar contrast to that ofthe target image can be secured, in an image subjected to the gradationcorrection, in a local region where brightness has varied drastically inthe target image which has not been subjected to the gradationcorrection. When the image data to be processed by the gradationcorrection is YUV data subjected to YUV conversion processing, theconfiguration of the image processor 52 has to be changed into aconfiguration in which a luminance component (Y) value of each pixel inthe target image is provided to the aforementioned ε-filter 103.

In addition, the effects obtained by use of the ε-filter 103 can be alsoobtained by using another smoothing filter in place of the ε-filter 103as long as the smoothing filter has edge holding performance. Forexample, another weighted average filter such as a bilateral filter canbe used as such a smoothing filter. If a bilateral filter or the like isused, similar effects to those by use of the ε-filter 103 can beobtained.

In addition, of the effects obtained by the gradation correction by theimage processor 52 as described above, the effect that similar contrastto that before the gradation correction can be secured in a localregion, where brightness has varied drastically in a target image, in animage subjected to the gradation correction, can be obtained if theimage processor 52 has a configuration satisfying the followingcondition. That is, the effect can be obtained fundamentally ifbrightness information in which brightness of each pixel has beensmoothed is included in brightness information of the pixel to beprovided to the corrected gain calculator 115 for calculating the gain(g_lev) of the pixel.

Accordingly, the ε-filter 103 constituting the image processor 52 can bereplaced, for example, by a typical low pass filter. That is, the imageprocessor 52 may be configured to use a typical low pass filter toperform smoothing processing on a brightness component image using thefirst brightness component value (max) as a pixel value of each pixel soas to acquire brightness information of each pixel of a global luminanceimage composed of low-frequency components of the brightness componentimage, that is, brightness information corresponding to the secondbrightness component value (max_ε). Even when the ε-filter 103 isreplaced by a typical low pass filter as described above, it is possibleto obtain the effect that similar contrast to that before the gradationcorrection can be secured in a local region, where brightness has varieddrastically in a target image, in an image subjected to the gradationcorrection.

In addition, in the image processor 52, the ε-filter 103 performsfiltering processing on the first brightness component values (max) ofall the pixels in the target image acquired by the RGB max computingunit 102 so as to generate the second brightness component values(max_ε) of all the pixels. However, it will go well if the secondbrightness component values (max_ε) can provide brightness informationexpressing the global brightness condition of the target image. Thefirst brightness component values (max) of all the pixels in the targetimage do not have to be provided to the ε-filter 103.

Accordingly, the image processor 52 may be, for example, configured asfollows. For example, the image processor 52 acquires first brightnesscomponent values (max) from the target image which has been oncereduced, so as to generate second brightness component values (max_ε) inthe reduced image. Then, the image processor 52 newly generates secondbrightness component values (max_ε) for pixels corresponding to theoriginal image size from the generated second brightness componentvalues (max_ε). If such a configuration is used for the image processor52, the time required for the filtering processing of the ε-filter 103can be shortened.

On the other hand, of the effects obtained by the gradation correctionby the image processor 52 as described above, the effect that goodgradation is secured in a region which has been dark in the target imagewhile the occurrence of a “whiteout” phenomenon is suppressed in aregion which has been bright in the target image can be obtained even ifthe gain (g_lev) of each pixel is not perfectly but schematicallyinversely proportional to the fourth brightness component value (mix).Accordingly, the aforementioned gain computing function is not limitedto the function shown in FIG. 9A, but, for example, a function shown inFIG. 9B or a function shown in FIG. 9C may be used.

In addition, of the effects obtained by the gradation correction by theimage processor 52 as described above, the effect that the probabilityof the “whiteout” phenomenon appearing in the pixels which have beenbrighter than their peripheral pixels in the target image is decreasedcan be obtained even if the configuration of the combination processingunit 114 is changed as follows. That is, the configuration of thecombination processing unit 114 may be changed so that processing forcombining the first brightness component value (max) with the thirdbrightness component value (adj) is performed only on pixels which arebrighter than some degree in the target image.

Accordingly, the combination ratio computing function used forcalculating the fourth brightness component value (mix) by thecombination processing unit 114 may be characterized, for example, asshown in FIG. 8B. That is, the combination ratio computing function usedby the combination processing unit 114 may be characterized in that thecombination ratio (α) of the first brightness component value (max) tothe third brightness component value (adj) increases in proportion tothe first brightness component value (max) when the first brightnesscomponent value (max) exceeds a predetermined combination threshold(“127” in the illustrated example).

Further, in order to obtain the effect that the probability of the“whiteout” phenomenon appearing in the pixels which have been brighterthan their peripheral pixels in the target image can be decreased, theconfiguration of the combination processing unit 114 may be changed asfollows. For example, the configuration of the combination processingunit 114 may be changed to generate the fourth brightness componentvalue (mix) by combining the first brightness component value (max) withthe third brightness component value (adj) uniformly at a predeterminedcombination ratio. Alternatively, for example, the configuration of thecombination processing unit 114 may be changed to generate the fourthbrightness component value (mix) by combining the first brightnesscomponent value (max) with the third brightness component value (adj) ata predetermined combination ratio which changes largely stepwise inaccordance with the magnitude of the first brightness component value(max).

In addition, the backlight effect level computing unit 111 of the imageprocessor 52 may compute the backlight effect level (gk_lev) using RGBdata of an image obtained by reducing a target image. In addition, anymethod for computing the backlight effect level (gk_lev) in thebacklight effect level computing unit 111 may be used. The backlighteffect level computing unit 111 may compute the backlight effect level(gk_lev) in another method than the aforementioned method.

In addition, in the image processor 52, the contrast adjustment unit 113corrects the second brightness component value (max_ε) in accordancewith the degree of backlight effect in the target image so as to obtainthe third brightness component value (adj) as described above. Thus, thecontrast in the image subjected to the gradation correction can beautomatically adjusted. However, the contrast in the image subjected tothe gradation correction may be changed automatically not always inaccordance with the degree of backlight effect in the target image but,for example, in accordance with the distribution condition of brightnessin the target image. More specifically, a functional part for acquiringa histogram indicating the distribution condition of brightness in thetarget image may be provided in the image processor 52 in place of thebacklight effect level computing unit 111, while a functional part fordetermining the contrast adjustment level (adj_lev) in accordance withpredetermined setting rules based on the histogram is provided in placeof the contrast adjustment level calculator 112.

In addition, in the image processor 52, the contrast adjustment level(adj_lev) can be adjusted to adjust the contrast in the image subjectedto the gradation correction. Accordingly, for example, the imageprocessor 52 may have a configuration in which a given contrastadjustment level (adj_lev) decided by the CPU 9 in accordance with arequest from a user of the digital camera 1 is provided to the contrastadjustment unit 113 in place of the backlight effect level computingunit 111 and the contrast adjustment level calculator 112.

In addition, in the gradation correction by the image processor 52, thecontrast adjustment unit 113 calculates the third brightness componentvalues (adj) of all the pixels by the following Expression (1).

adj=adj_lev×(max_ε−max)+max   (1)

That is, the contrast adjustment unit 113 corrects the second brightnesscomponent values (max_ε) of all the pixels using the same expression.However, the contrast adjustment unit 113 may be configured to applydifferent expressions to pixels which are brighter than their peripheralpixels in the target image, that is, pixels expressed by“(max_ε−max)>0”, and pixels which are darker, that is, pixels expressedby “(max_ε−max)<0”, respectively, to calculate the third brightnesscomponent values (adj).

For example, the contrast adjustment unit 113 may be configured tocalculate the third brightness component values (adj) for the brighterpixels using the following Expression (1-a):

adj=adj_lev×(max_ε−max)+max×ks   (1-a)

and calculate the third brightness component values (adj) for the darkerpixels using the following Expression (1-b):

adj=adj_lev×(max_ε−max)+max×kt   (1-b)

That is, the contrast adjustment unit 113 may be configured to apply theexpressions only different in coefficients ks and kt to the brighterpixels and the darker than their peripheral pixels respectively tocalculate the third brightness component values (adj). When the thirdbrightness component values (adj) are calculated for the brighter pixelsand the darker pixels than their peripheral pixels individually, thecontrast in the image subjected to the gradation correction will beoften improved if the coefficient ks in the Expression (1-a) is set tobe larger than the coefficient kt in the Expression (1-b).

In addition, according to this exemplary embodiment, the gain settingprocessor 104 is configured to be constituted by the backlight effectlevel computing unit 111, the contrast adjustment level calculator 112,the contrast adjustment unit 113, the combination processing unit 114and the corrected gain calculator 115. However, the gain settingprocessor 104 may be configured in the following manner so that similarcontrast to that in the target image can be secured in a local regionwhere brightness varies drastically in the target image when thegradation correction is performed.

For example, the gain setting processor 104 may be configured so thatthe combination processing unit 114 is removed, and the third brightnesscomponent value (adj) is provided to the corrected gain calculator 115as it is. Alternatively, the gain setting processor 104 may beconfigured so that the backlight effect level computing unit 111, thecontrast adjustment level calculator 112 and the contrast adjustmentunit 113 are removed, and the second brightness component value (max_ε)is provided to the combination processing unit 114 as it is.Alternatively, the gain setting processor 104 may be configured so thatall the units other than the corrected gain calculator 115 are removed,and the second brightness component value (max_ε) is provided to thecorrected gain calculator 115 as it is.

In addition, the exemplary embodiment of the invention has beendescribed here on the digital camera 1 including the image processingapparatus according to the invention. However, the invention can be alsoapplied to another imaging apparatus than the digital camera 1, forexample, an imaging apparatus having a configuration capable ofrecording moving pictures. The invention can be applied to variousimaging apparatuses, for example, including a digital camera providedwith a CMOS (Complementary Metal Oxide Semiconductor) type solid-stateimage sensing device, a digital camera capable of picking up movingpictures as well as still pictures, a digital video camera principallyserving to pick up moving pictures, etc., in addition to CCDs.

In addition, the invention is not limited to an imaging apparatus, butcan be also applied to an image processing apparatus which performsimage processing on images stored as image data in a desired storagemedium. An example of the image processing apparatus includes a printerfor printing an image based on image data.

According to the invention, the image processor 52 shown in FIG. 2 canbe, for example, implemented by an ASIC (Application SpecifiedIntegrated Circuit), or a CPU of a desired computer, a memory, a programloaded on the memory etc. Further, a computer storage medium for storingprograms for executing the operation described in the aforementionedexemplary embodiment may be used.

While the present invention has been shown and described with referenceto certain exemplary embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims. It is aimed, therefore, to cover in theappended claim all such changes and modifications as fall within thetrue spirit and scope of the present invention.

1. An image processing apparatus comprising: a brightness informationacquisition unit configured to acquire brightness information indicatingbrightness of each pixel in a target image, wherein high-frequencycomponents are eliminated from the target image; a correctionmagnification setting unit configured to set, for each pixel of thetarget image, a correction magnification based on the brightnessinformation, wherein the correction magnification is substantiallyinversely proportional to the brightness of the pixel; and a gradationcorrection unit configured to correct the brightness of each pixel basedon the correction magnification.
 2. The apparatus according to claim 1,wherein the gradation correction unit is configured to correct aplurality of color component information belonging to each pixel basedon the correction magnification so as to correct the brightness of eachpixel.
 3. The apparatus according to claim 1, wherein the brightnessinformation acquisition unit comprises: a brightness componentextraction unit configured to extract brightness components from thetarget image; and a smoothing unit configured to smooth a brightnesscomponent image composed of the brightness components.
 4. The apparatusaccording to claim 3, wherein the smoothing unit is a smoothing filtercapable of holding edges in the brightness component image.
 5. Theapparatus according to claim 4, wherein the smoothing filter adjusts apixel value of each of peripheral pixels around an pixel of interestsuch that a difference between a pixel value of the pixel of interestand the pixel value of each peripheral pixel is less than or equal to athreshold value, and sets an average value of the pixel value of thepixel of interest and the pixel values of the peripheral pixels as a newpixel value of the pixel of interest.
 6. The apparatus according toclaim 2, wherein the correction magnification is inversely proportionalto the brightness of each pixel.
 7. The apparatus according to claim 2,wherein the correction magnification setting unit comprises: acorrection unit configured to correct the brightness of each pixelindicated by the brightness information in accordance with thebrightness of each pixel in the target image, and wherein the correctionmagnification setting unit is configured to set, for each pixel, acorrection magnification inversely proportional to the brightness ofeach pixel corrected by the correction unit.
 8. The apparatus accordingto claim 7, wherein the correction unit comprises: a combination unitconfigured to combine the brightness of each pixel indicated by thebrightness information with the brightness of each pixel of the targetimage at a given combination ratio, so as to correct the brightness ofeach pixel indicated by the brightness information, and wherein thecorrection unit is configured to acquire the brightness of each pixelcorrected by the combination unit as corrected brightness.
 9. Theapparatus according to claim 8, wherein the correction unit furthercomprises: a combination ratio determination unit configured todetermine the given combination ratio in proportion to the brightness ofeach pixel in the target image.
 10. The apparatus according to claim 7,wherein the correction unit further comprises: a subtractor configuredto subtract the brightness of each pixel of the target image from thebrightness of each pixel indicated by the brightness information; amultiplier configured to multiply the brightness of each pixel obtainedby the subtractor, by a given contrast adjustment value; and an adderconfigured to add the brightness of each pixel of the target image tothe brightness of the pixel obtained by the multiplier, and wherein thecorrection unit is configured to acquire the brightness of each pixelobtained by the adder as corrected brightness.
 11. The apparatusaccording to claim 10, wherein the correction unit further comprises: abacklight effect degree acquisition unit configured to acquire a degreeof backlight effect of the target image; and an adjustment valueacquisition unit configured to acquire an adjustment value proportionalto the degree of backlight effect as the given contrast adjustmentvalue.
 12. The image processing apparatus according to claim 7, whereinthe correction unit comprises: a subtractor configured to subtract thebrightness of each pixel of the target image from the brightness of eachpixel indicated by the brightness information; a multiplier configuredto multiply the brightness of each pixel obtained by the subtractor, bya given contrast adjustment value; an adder configured to add thebrightness of each pixel of the target image to the brightness of thepixel obtained by the multiplier; and a combination unit configured tocombine the brightness of each pixel obtained by the adder with thebrightness of each pixel of the target image at a given combinationratio, and wherein the correction unit is configured to acquire thebrightness of each pixel corrected by the combination unit as correctedbrightness.
 13. A computer-readable medium storing a program for causingthe computer to perform operations comprising: (a) acquiring brightnessinformation indicating brightness of each pixel in a target image,wherein high-frequency components are eliminated from the target image;(b) setting, for each pixel of the target image, a correctionmagnification based on the brightness information, wherein thecorrection magnification is substantially inversely proportional to thebrightness of the pixel; and (c) correcting the brightness of each pixelbased on the correction magnification.